Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for instantiating Half-Typed-grid from stored float or double grids #1780

Draft
wants to merge 1 commit into
base: feature/half_grid_support
Choose a base branch
from

Conversation

konivo
Copy link

@konivo konivo commented Mar 22, 2024

This is still WIP and it depends on #1730 .

The main motivation for this patch is to be able to load half-typed grids directly from files containing non-half-typed grids in order to keep the process peak memory low. When having large grids or large number of grids this can be important.

The PR does (atm) the conversion according to the following rules:

stored grid value type desired conversion returned grid
float half half
double half half
vec2f half vec2h
vec3f half vec3h
vec2d half vec2h
vec3d half vec3h

The above rules can be extended if desirable, and in the future there could be also 'store' counterpart to do conversion into other grid types while storing. That could potentially replace the need for "saveFloatAsHalf" flag.

The core implementation has been already tested with various vdb files, some of them using saveFloatAsHalf, and the approach seems to be working without issues.

The implementation is straightforward and follows these steps:

  • User specifies desired conversion when calling File::open or when instantiating Stream
  • Desired conversion information then propagates into StreamMetadata
  • GridDescriptor::read picks up the conversion from the metadata and from io::ConvertingReaderFactory it gets a reader and the resulting grid valuetype. It then instantiates a grid with this new valuetype instead of using the original value type. If there is no suitable reader for the desired conversion, the method instantiates a grid with the original valuetype.
  • The reader returned from io::ConvertingReaderFactory is later also stored in StreamMetadata and used during topology/buffers loading.

Copy link

linux-foundation-easycla bot commented Mar 22, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: konivo / name: Ivo Kondapaneni (c292c28)

@konivo konivo changed the base branch from master to feature/half_grid_support March 22, 2024 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant